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quantity of systems engineers and poor quality systems engineering is credited as central to 
program failure. In an Army Systems Engineering Forum, the Army System of Systems 
Engineer (SOSE) asked what could be done to recruit, train, certify, and retain systems 
engineers. This paper answers that question, and identifies that it cannot be “fixed” in 
isolation of addressing an Army culture that does not focus its efforts on training the 
personnel it already has. Quantity issues are not being addressed at the service level with 
recruiting efforts. Organizations do not have formal collateral personnel exchange 
programs, yet many perform systems engineering functions. Training and certification 
gaps exist despite availability of training because personnel are not mandated to be certified 
to accept positions, in many cases. Systems engineering, although not blameless, is not the 
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I. INTRODUCTION 


A. BACKGROUND 


Arguably, the United States (U.S.) fields the most operationally effective military 
force in the world. However, fielding such a force has been challenging, as seen by the 
multiple Government Accountability Office (GAO) reports of cost and schedule 
overruns. According to the GAO, development costs for Major Defense Acquisition 
Programs (MDAP) are often underestimated at program initiation, sometimes by 30 to 
40% (GAO, 2008c). Additionally, weapons systems programs are initiated without 
sufficient knowledge about system requirements, technology, or design maturity. This 
lack of knowledge leads managers to rely on assumptions that are consistently too 
optimistic, exposing programs to unnecessary risks, and, ultimately, cost growth and 
schedule delays (GAO, 2008b). The GAO has also reported that within the Department 
of Defense (DoD), there was an average delay of 22 months in delivering initial 


capabilities for MDAPs (GAO, 2010). 


The acquisition community within the DoD has come under intense scrutiny from 
Congress for cost overruns and schedule delays and has caused extreme frustration for 
the warfighters because of the late-to-need delivery of reduced capabilities (GAO, 2009). 
The increasing complexity of acquisitions within the DoD is part of the reason. Weapons 
systems acquisitions, the totality of effort to bring a product to fielding, are no longer 
complete stand-alone fielded entities; instead, they are systems within systems with 


interdependencies on a scale never before attempted. 


The U.S.—and specifically the DoD acquisition process—faces a complex and 
uncertain security landscape in which the pace of change continues to accelerate. 
Changes include new foreign powers, non-state actors, and the availability of destruction 


enabling technologies (DoD, 2010). 


The difficult task of a systems engineer includes translating the warfighter’s 
request for capability into a solution that properly addresses the tradeoffs between 


multiple factors (e.g., cost, schedule, performance, and quality). This includes the 


1 


interconnectedness of subcomponents and their impact on the system within other 
systems. Internal reviews and external studies have postulated that the quantity and 
verifiable quality of systems engineers present in the government workforce is not equal 
to this task (Gates, et al., 2009). The quality of a systems engineer, for the context of this 
paper, is defined as the measure of a person’s ability to apply the tools and best practices 


of systems engineering consistently with success in the execution of their duties. 


The lack of quality and proper systems engineering early! in system design results 
in waste. At best, it causes cost growth and time delays. At worst, it results in unusable 


products and/or cancelled programs (Defense Acquisition University [DAU], 2011). 


This complex and uncertain security landscape has been identified as a significant 
problem by a 2009 GAO report, which identified knowledge gaps that are largely the 
result of a lack of early and disciplined systems engineering analysis of a weapon 
system’s requirements prior to beginning system development (GAO, 2009). The 2009 
GAO report also states that the government often does not perform the proper up-front 
requirements analysis to determine whether the program will meet its needs, significant 
contract cost increases can and do occur as the scope of the requirements changes or 


becomes better understood by the government and contractor (GAO, 2009). 


Since the early 2000s, the DoD and the Department of the Army (DA) have seen a 
dramatic deterioration in the capability to field weapons systems on the planned budget, 
cost, and schedule (GAO, 2009). Current military acquisition programs take two to three 
times longer to move from program initiation to system deployment than they did 30 
years ago (Air Force Studies Board [AFSB], 2008). This systematic delay has occurred 
during a period in which traditional threats have been increasing in frequency and 
emergent threats in cyber, electromagnetic, and chemical/biological warfare are being 
implemented at a more rapid pace. Many causes for this trend have been suggested, 
including the increased complexity of the tasks and the systems involved from both 


technological and human/organizational perspectives; funding instability; loss of 


! Early is defined as starting at the formulation of the initial concept for a program. 
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“mission urgency” after the end of the Cold War; bureaucracy—which increases cost and 
schedule but not value—and the need to satisfy the demands on an increasingly diverse 


user community (AFSB, 2008, p. 1) 


Figure 1 provides a visual perspective of how the acquisition landscape has 


evolved and what we can expect for the next decade (Torelli, 2010b). 
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Figure 1. _ Perspective for the Next Decade (From Torelli, 2010b) 


The Assistant Secretary of the Army (Acquisition, Logistics, and Technology) 
(ASA[ALT]) considers the systems engineering expertise within the Army workforce 
fundamental to delivering on-time, on-budget, and on-performance products. This 
assessment is supported by the 2010 GAO annual report on defense acquisition, which 
states that the GAO analysis allows them to make observations about DoD’s management 
of technology, design, and manufacturing risks and its use of early systems engineering 


to reduce these risks (GAO, 2010). Because the scope of projects has grown from 
a 


single-use systems to a federated systems of systems, ASA(ALT) believes that an 
increase in the amount of systems engineering capability within the Army would 
dramatically increase the percentage of projects that would be delivered on time and on 
budget, while also meeting original key performance parameters. In a 2009 RAND 
Corporation study performed for the ASA(ALT), researchers observed that the 
underlying problem in major acquisition programs is a lack of systems engineering 
expertise overall and a lack of effective systems engineering in system development 


started as early as the requirements development phase (Gates et al., 2009). 


The focus of this paper is on systems engineering within the context of Army 
acquisition. More specifically, we explore systems engineering staffing practices 
(recruit, train, and retain) within the Army Acquisition Corps, and the perception that the 
systems engineering workforce is either a source of, or solution for, program failures 
attributed to acquisition complexity. Development of a useable, viable framework for 
systems engineering usage across the complete DoD acquisition process will be a 
significant challenge for the Army due to the complex nature of Army acquisition 
programs. Our purpose in this project was to identify weaknesses in DA’s systems 
engineering staffing and personnel approaches, to determine methods for identifying and 
addressing shortfalls, assess temporary and long-term needs, and to determine potential 


policy changes necessary to maintain a quality systems engineering capability. 


B. MOTIVATION FOR THIS PROJECT 


The ASA(ALT) SoSE staff believes: 


e the Army needs to increase the overall strength of its systems engineering 
capabilities; 
e the SoSE needs to make a recommendation to leadership for supporting 


this increase; and 


e the recommendation must articulate recruitment, training, certification, 
retention, and cross-program utilization; and it should contrast where and 
how systems engineers are used currently for background. (M. Kwinn, 
personal communication, July 13, 2010) 


These capabilities will be used in 


e Acquisition 

e Requirements development 

e Test and evaluation 

e System of systems integration 

e Personnel recruiting, training, and retention 


The ASA(ALT) is committed to determining the best way to recruit, train, and 
retain systems engineers to address this issue, but he also wants to know if the lack of 


systems engineers is the only problem. 


The central question is: what recommendations should the ASA(ALT) SoSE 
make to the Under Secretary of Defense (Acquisition, Technology and Logistics) 
(USD[AT&L]) to ensure that the proper personnel are recruited, trained, certified, and 
retained to increase the U.S. Army systems engineering capability that is needed to meet 
the increasingly complex requirements of the Army’s system of systems strategy? For 
example, how does the DoD systems engineering community ensure that the proper skill 
sets are being identified and implemented correctly within the systems engineering 
community to ensure a qualified and retainable acquisition, logistics and technology 


(ALT) workforce? 


C: RESEARCH QUESTIONS 


The questions we extrapolated from the focus of the ASA(ALT) include the 


following: 

e Can systems engineering help the Army acquire products that meet 
requirements, on cost, and on schedule? 

e What are the barriers for the Army in acquiring products that meet 
requirements and satisfy constraints of cost, schedule, and policy? 

e Are programmatic errors, that are not the sole responsibility of systems 
engineering, being attributed to systems engineering rather than poor 
program management? 

e Is the lack of a formalized systems engineering approach within the Army 


causing Army acquisition programs to fail? If so, what can be done to 
resolve this? 


How does the Army formalize its systems engineering approach in 
acquisition programming to ensure that Army acquisition programs are 
positively affected by systems engineering personnel? 


How does the Army set up a systems engineering career path that allows 
both traditional engineers and systems integrators to succeed? 


Are there additional skill sets that should be incorporated into the current 
systems engineering path, that would allow for less technical (but still very 
capable) individuals with a management focus to function in the systems 
engineering career field? 


How can the DA benefit from what other DoD organizations are doing to 
implement systems engineering? 


ORGANIZATION OF THE REPORT 


Chapter I provides a background, explains our motivations, and creates the 


starting point for the questions that are core to our research. 


Chapter II provides our analysis approach. 


Chapter III analyzes whether systems engineering is the central issue that external 


studies postulate, or if there are other contributing factors. 


Chapter IV provides a review of the current state of systems engineering with 


additional focus on the Army’s needs. 


Chapter V assesses our research and details our findings and results. 


Chapter VI provides our conclusions, and makes recommendations for changing 


the structure and processes for building a systems engineering community to meet the 


needs and expectations of 21st century Army acquisition programs and stakeholders. 


Il. ANALYSIS APPROACH 


A. ANALYSIS STRUCTURE 


The starting point of view for our case was from the position of the most senior 
engineering advisor (SoSE) to the Army director of acquisition. The SoSE has previous 
work experience that includes serving as the Army’s chief architect in the Army’s Deputy 
Chief of Staff for Command, Control, Communications, Computers, and Intelligence and 
Director of the Army’s Central Technical Support Facility (CTSF) at Fort Hood, TX. 
The context of this perception is where this question of “best method” for recruiting, 
training, and retaining systems engineers originates. In parallel, we asked: “why is 
recruiting, training and retaining the perceived solution and what problem(s) will this 
solve?” This point of view from the SoSE is greatly influenced by his personal 
experiences. A future SoSE might have a different point of view due to personal 
experiences, but this point of view is critical because it comes from the peak of the 


Army’s engineering expertise. 


To understand the intention and subtext of the question, we have extrapolated 
additional questions, as shown in Figure 2. The majority of these sub-questions can be 
reached by asking “why?” or “what makes this so?” Using questions of this type as a 
tool, we focused our research on the perceived positive impact that greater numbers of 
highly qualified systems engineers would have on acquisition programs, rather than on 


the quality of the currently trained DoD systems engineers. 
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Figure 2. Problem Deconstruction 


The desired end state is a successful integration of systems engineering processes 


on individual programs that result in a quality and cost effective system of systems. This 


integration has several components, some less obvious than others: 


Successful programs need effective systems engineering to integrate their 
components into a functional system. Early initiation of systems 
engineering into the acquisition process helps to assure efficiency, reduce 
overall cost, and increase the chances of staying on schedule. However, 
this can prove to be costly, both in terms of funding and time. Early in the 
acquisition process PMs may be more concerned with more tangible 
results (boots on the ground) in order to maintain the funding stream for 
their program. 


Successful integration of products from multiple PMs, requires effective 
systems engineering in the beginning and the middle of programs to 
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increase the likelihood that PMs will buy into and work toward a shared 
goal. It often requires creativity in adapting systems to achieve more than 
the sum of the individual components. It can also require some shifting of 
responsibilities and costs between programs to achieve the “best” effect. 
Process standards clearly fall within the realm of effective systems 
engineering. The shifting of any responsibility or cost between PMs 
requires management skills more than engineering skills. 


e Successful integration also requires a working level of interoperability 
between supporting organizations. Without interoperability between 
organizations, test and evaluation of the interdependent products to assess 
interface standards for compliance, or possibly for modification, is 
problematic at best. This ability is often described as “herding cats,” and 
has more of a political or financial emphasis than pure engineering. 


As indicated by the above list, it is apparent how skills move from classical 
engineering to adaptive expertise with an engineering focus, and on to leadership or 


governance functions with an overall acquisition focus. 


One of the difficulties in presenting a definition of systems engineering in concise 
terms, can be found in the relational differences that a single systems engineering 
definition can have from different points of view. In other words, systems engineering 
can mean different things to different organizations, and can have divergent meanings to 


the people within those organizations. 


The analysis then moved outside of the frame of reference of the senior advisor, 
or SoSE, to encompass alternate points of view from successively different organizations 
and institutions, in order to draw comparisons. We reviewed documents, briefings, white 
papers, and training materials from the Army, Air Force, Navy, National Aeronautics and 
Space Administration (NASA), and several DoD industrial partners on systems 
engineering practices within their respective organizations. We examined and reviewed 
these documents with curricula from several educational institutions. A significant 
correlation was found in certification, experience, duties, expectation, and education of 
systems engineers. Consistency would have been a strong indicator for a “shared” vision 


or understanding of what the systems engineer would be doing in each organization. 


Chapter III will analyze how the central question asked by the SoSE could have 
been formulated in error due to the requestor’s position, organization, and background. A 


comparison is provided between the technical, organizational, and personal perspectives. 


B. LITERATURE REVIEW 


Our review of the literature encompassed the areas of interest that we identified as 
our research questions, and those areas that we further detailed and highlighted in our 
research matrix (Appendix A). Research for the thesis project focused on reviews of 
Army and other Service policy statements on systems engineering, Army and other DoD 
systems engineering organizational websites, and a variety of university curricula, 
International Council on Systems Engineering (INCOSE), and other systems engineering 
certification organizations. Additionally, in the area of human capital, recruitment, and 
retention, we reviewed workforce surveys and programs from NASA, a large-scale 
organization similar to the DoD, and resources from the Office of Personnel Management 
(OPM) Human Capital Assessment and Accountability Framework, which provided 
information on recruitment and retention. We also reviewed information gleaned from 
our coursework during the Naval Postgraduate School Master of Science in Program 


Management (MSPM) program. 
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Hl, ISSYSTEMS ENGINEERING THE CENTRAL ISSUE? 


A. PRIMARY AND CONTRIBUTING QUESTIONS 


1, Primary Question 
e How does the Army recruit, train, certify, and retain qualified systems 
engineers? 


The ASA(ALT) directly experiences the combined effects of the outcome rather 
than a lack of systems engineering in isolation. The question of how to recruit, train, and 
retain systems engineering personnel is deceptively straightforward, or would seem so 
until the answer becomes “it depends.” How this question is answered, it turns out, 
depends a great deal on how these systems engineers are expected to perform after they 


have been recruited and trained. 


Zs Contributing Questions 


The answer to the primary question leads immediately to the following 


contributing questions: 


e What makes a systems engineer qualified? 
e Why are systems engineers perceived to be in short supply? 
e Is a lack of systems engineers the only problem, or is that lack part of a 


more complex issue? 


To provide the answers that have the greatest possible impact, context 
surrounding the reason for why a lack of qualified systems engineers is believed to 
matter, must be explored. The primary question, therefore, is too broad reaching to be 
met with a succinct answer that will satisfy all of the challenges facing the Army 
acquisition community. Each of the subsets of the primary question is narrow enough 


when asked individually to provide a slightly more succinct answer. 


B. WHAT IS A SYSTEMS ENGINEER? 


The DoD defines systems engineering as an interdisciplinary approach or a 


structured, disciplined, and documented technical effort to simultaneously design and 
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develop systems products and processes to satisfy the needs of the customer. Systems 
engineering transforms needed operational capabilities into an integrated system design 
through concurrent consideration of all life cycle needs (DAU, 2010b). The GAO 
defines systems engineering as the translation of customer needs into specific product 
requirements for which requisite technological, software, engineering, and production 
capabilities can be identified through requirements analysis, design, and testing (GAO, 


2009). 


To begin to understand the problem, we first had to understand the current area(s) 
of operation under which systems engineering were expected to perform. Before 
beginning to answer a question, that question must be understood. Context is critical 
here. Before we gathered data exclusively in support of recruitment, training, and 
certification programs, we needed to inquire why this question was being asked. As 
stated earlier in this paper, a deconstruction of the problem(s) was used to make sure that 
we were researching the right questions in the right context. This approach may seem 
obvious, but unfortunately making sure the right question is being asked can lead to 
discovery of underlying context—the intent of the question should not be lost in the 
wording. A child asking “why does my stomach hurt?” could prompt one of several 
reasons: illness, hunger, overeating, or roughhousing with a sibling. Too many words 


have multiple meanings, and sometimes the question needs a bit of research. 


Despite today’s bleak economic outlook, there are glimmers of opportunity and 
growth in the technology and engineering industries—and systems engineering is 
emerging as a must-have career field. According to a ranking of the best jobs in America 
by CNN Money, “there will be a high demand for systems engineers over the next 
decade” (Amaba, 2010). In his article, Ben Amaba (2010) stated that: 

The role of today’s systems engineer combines the best attributes of 

electrical engineers, mechanical engineers, and software developers to 

take on the world’s most challenging problems. These types of challenges 

also come with a high level of uncertainty and risk, which adds another 

unique layer of skill requirements to the job. (Amaba, 2010, p.1) 

He went on to state that to help meet the growing demand for systems engineers, a 


new generation of specialists will be needed. And with the retirement of the “Moonshot 
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Generation,” the engineering experts who were the driving force in successfully landing 
man on the moon, the push to replenish these ranks is all the more urgent. Thankfully, an 
increasing number of colleges and universities are evolving their engineering curriculum 


to address this need. 


Systems engineering is a discipline that emphasizes best practices across multiple 
disciplines. The systems engineering process is considered reusable; however, it is 
dependent on having the necessary expertise with the pertinent historical knowledge to 
recognize the good and bad from previous systems engineering process efforts. In an 
ideal situation, the personnel undertaking the systems engineering process would have 
requisite knowledge through previous practical experience. During an April 7, 2010 
keynote speech, Dr. Art Pyster (2010), of the Stevens Institute of Technology, posited 
that previous practical experience is rarely available at the level necessary to provide 
adequate systems engineering guidance. When practical experience is not readily 
available, the systems engineering process must normally default to the academic training 
realm, in which theoretical knowledge is imparted on the systems engineering students 
with the expectation that an extraction to the practical systems engineering process arena 
will occur. This background of practical experience is referred to as the difference 
between classical engineering and adaptive engineering?. Some of this theoretical 
knowledge is imparted in the form of education in critical thinking and problem solving, 
which comes with the process of learning to become an engineer. This foundation is built 
upon in order to gain the experimental knowledge and understanding of the systems 


engineering concept in the context of an entire system. 


Cc; MULTIPLE PERSPECTIVES 


1. A Multiple Perspective Approach on Why the Army May Have Asked 
the Wrong Questions 


While identifying the lack of systems engineering as the cause and programmatic 


failure as the effect, the ASA(ALT) leadership may have been using an overly technical 


2 Adaptive engineering is defined as the process whereby an item is modified to meet the requirements 
of a user that the item was not originally designed for. 
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perspective. An organizational/institutional perspective would ask What must be done? 
and Who must do it? Further contrast is provided by the personal/individual perspective, 
which would instead seek to identify factors that drive the individual to do something 


about the situation. What empowers the individual, rather than the organization? 


In his article, Harold A. Linstone (1984) tells us that a multiple perspective 
approach links the technical/analytic perspective (T) with organizational/institutional (O) 
and personal/individual (P) perspectives. The approach is to use T, O, and P together. It 
also helps to explain why decision makers cannot rely on a single perspective alone. 
Linstone says: 

The T perspective: Problems are simplified by abstraction, idealization, 

and isolation from the real world. The implicit assumptions and 

characteristics include reductionism, reliance on scientific logic and 

rationality, problem-solution focus, quantification, use of data and models, 
optimization, and objectivity of the analyst. Jay Forrester's systems 
dynamics modeling of companies, cities, and the world is an example. 

The power and success of this technical world view and its value in 

yielding remarkable insights and excellent predictions in science and 

engineering remains unchallenged. But, as the recent work in complexity 
science has underscored, it has serious limitations in dealing with 
complex, nonlinear, adaptive systems. Unfortunately, most real world 

socio-technical systems are of this kind. (Linstone, 1984, p. 1) 

The primary question taken alone appears to assume that addressing the vacancies 
in systems engineering personnel and the requisite systems engineering skills, meaning 


certification, will resolve the problems facing the Army acquisition community. 


In a systems engineering forum, where the ASA(ALT) gathered subject matter 
experts in order to gain an acknowledged consensus, a concern was raised regarding the 
methods used for decades in developing weapons, platforms (trucks or tanks), 
communications, and other tools of war and peace, and whether they were adequate 


enough to ensure a fully functional, integrated capability in the hands of the warfighter. 


Systems are now both interdependent and, at times, in competition for resources 
like power, space, and weight on their respective platforms. Among many examples for 
how the big picture was lost by developers of individual components, was the following 


example given at the former Future Combat System (FCS) synchronization summit. 
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The platform (in this case, an armored troop carrier) was slated for the installation 
of more computer equipment, communications gear, and electronic warfare defense 
ability than it was able to physically fit or electrically power (Joint Program Executive 
Office [JPEO] Joint Tactical Radio System [JTRS], 2010). Based on researcher Alan 
Clayton’s experience with the JTRS program, we were able to determine that the 
platform was a system of systems. The systems were developed in isolation with each 
PM assuming resource availability. The first equipment fielding into that vehicle would 
deplete most of the resources, leaving less than adequate space or power for the 


remaining components of the system of systems. 


Ze Multi-Organizational Interaction Point of View 


When challenged with hardware and software conflicts, a Program Executive 
Office (PEO) must decide whether to rewrite the software or fix the problem in hardware. 
The PM responsible for the software may have a strong opinion of the relative merits in 
the comparison that would not be shared with the hardware PM. Each PM may wish for 
the other to sacrifice funding, timeline, and program credibility rather than volunteer to 
take on the task. For programs within the Army or under a single PEO that were intended 
to operate together as a system of systems at program inception, there should be 
performance specifications that mandate one or the other PM to comply with the interface 
standards when known and risk management strategies implemented for unknown 


situations. 


This matters significantly, because from the point of view of a PM, success is 
usually specified internally as being within defined performance parameters—being on 
cost and on schedule. External factors, such as the change of an external interface, are 
considered risks to be managed. An organization considers external interoperability in 


terms of risk to program execution. 


3. Why the Organizational Perspective Matters 


Each organization and its respective PM have to interpret tasks within the context 
in which they are assigned and resourced. This means that their development is supposed 


to include knowledge of everything that will need to be done both within and on the 
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periphery of their acquisition effort. For example, the developer of a software application 
would need to understand the hardware and operating environment in which he will 
implement his program. There is at present no common software operating environment 
in use on all Army systems, so each application tends to be uniquely tailored and 
somewhat non-portable. If a change to an operating environment renders a second 
organization’s software application inoperative, the second organization’s perspective is 
not going to be in agreement with the organization that changed the operating 


environment. 


For programs allowed to gestate in the absence of a larger interface context or end 
state—connecting the dots once they mature will not only be difficult and expensive, but 
also will also require the use of management reserves to make necessary product changes. 
Worse yet, it may be open to interpretation as to whether the work is within scope and it 
may be hard to figure out how to legally expend funds. This interpretation is both a 
systems engineering issue and a contracts issue. The systems engineers from both parts 
of the future system (in the case of a two-part system), together with architects, work out 
the engineering issues, which are resolvable in trade-space. Decision makers work 
through the trade-space analysis and make the “big picture” political decision. The 
contracting person(s) carry out the consensus view. This is something NASA does 


routinely. Systems engineers also do this regularly in the commercial world. 


Organizations need to ensure that they do both the engineering for their product 
and manage the systems engineering for the product’s placement in the big picture. But 
who is in charge of the bigger picture? For example, the Army’s tactical network is a 
federated system of systems at best, which was designed using a systems engineering 
architectural process. Control, as such a term makes sense in the context of standards, is 
shared by multiple Army and Joint PMs and strongly influenced by commercial, federal, 
and DoD standards bodies—all while being directed by Army Staff elements that have 
the ability to influence decisions, if not directly, by control of personnel or funds. The 
design elements under the purview of an engineer or systems engineer need to be handled 
more effectively and qualified recruitment and qualified training can address those. 
However, engineering practices are not adequate to control all aspects of making 
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programs successful. The engineers in the PM shops need to follow requirements set 
forth by leadership in external organizations, but this requires an O perspective and 
engineers are very T perspective focused. Just as the T is the realm of the engineer, the 


PM must take responsibility for the O. 


D. REPEATED ATTEMPTS TO ‘FIX’ ACQUISITION AND SYSTEMS 
ENGINEERING 


There have been countless new and revised processes implemented through the 
DoD acquisition community over the past 10 years, yet there is almost no noticeable sign 
that systems development is becoming more efficient (GAO, 2009). The government 
trend in systems acquisition of over-budget and over-schedule programs is one of 
diminishing returns as the procurement of a system matures and the processes within the 
system become more complex. In a May 2010 Defense News article summarizing a 
recent GAO Report to the House Oversight and Government Reform Subcommittee on 
National Security, Rep. John Tierney, D-Mass., said the Pentagon is still not taking 
"some common-sense steps" (Matthews, 2010) that would almost certainly save money, 
such as testing prototypes to make sure they meet military needs before beginning 
production. Delays and cost increases have been persistent for decades, and have been 
"implicitly accepted as the cost of doing business. It will take considerable and sustained 


effort [to change that status quo]" (Matthews, 2010). 


Numerous efforts to reform the acquisition system have been undertaken by DoD, 
such as the many changes made to acquisition policies, as well as the recommendations 
made for improving the DoD’s acquisitions by various commissions, think tanks, and 
government organizations all of which culminated in various legislation passed by 
Congress. In 1986, the Packard Commission, named for its chair, Mr. David Packard, 
was appointed by President Ronald Reagan to study government procurement within the 
DoD. The culmination of this commission’s study resulted in the passage of the 


Goldwater-Nichols Act of 1986. Additionally, the Defense Acquisition Workforce 
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Improvement Act (DAWIA) of 19903, the Acquisition Streamline Act of 1994, the 
Clinger-Cohen Act of 1996, the Intelligence Reform and Terrorism Prevention Act of 
2004, and the Weapons Systems Acquisition Reform Act of 2009 were all passed by 


Congress to address improvements to the DoD acquisition process. 


In an effort to address cost and schedule overruns, the DoD has published 
numerous policies, undertaken many studies, and developed several guides and 
pamphlets, such as the DoD Instruction 5000.02, the Systems Engineering Guide for 
Systems of Systems (Director, Systems and Software Engineering, 2008), and a DAU 
Acquisition Encyclopedia entry, Systems Engineering Plan (DAU, 2009). The Naval 
Systems Engineering Technical Review Handbook (Department of the Navy, 2009), and 
the Air Force Systems Engineering Model (AF Center for Systems Engineering, 2010) 
are examples of what the other services have published to augment the DoD’s policies 
and to develop Service-specific processes. There is no equivalent document that 


currently exists within the DA. 


On December 8, 2008, the DoD issued an updated DoD Instruction 5000.02 
USD(AT&L), 2008) that included a number of major systemic changes, such as: an entire 
section on systems engineering; a requirement for a lead systems engineer to be placed on 
every PEO staff, a mandatory requirement for competitive prototyping, an increased 
emphasis on scheduling and executing timely systems engineering and technical reviews; 
and a requirement that all programs go through a Materiel Development Decision process 


prior to entering the acquisition system. 


Programs may fail or exhibit cost and schedule overruns for many reasons. Some 
of these are external to the program, such as funding instability, and others are internal to 
the program and, thus, under the control of DoD managers. Two critical factors that fall 
in the latter category and that relate to the success or failure of programs are the need for 
high-quality systems engineering and the related issue of the need for a high-quality 


systems engineering workforce. 


3 Extensive changes were made to the DAWIA in 2003, and the changes have been informally called 
DAWIA II, even though its Public Law number was never changed from the original numbering from 
1990. 
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With budgets becoming tighter, public scrutiny becoming stronger, increasing 
focus being placed on advanced technology, and demands arising from the shift toward 
network-centric warfare, there has been a major emphasis placed on systems engineering 
within the DoD (Wynne & Schaeffer, 2005). In addition to the previously referenced 
policies, the creation of the Systems and Software Engineering Office within the Office 
of the Under Secretary of Defense for Acquisition, Technology and Logistics 
(OUSD[AT&L]) point to an understanding of the contributions that systems engineering 
can make to modern acquisition. Multiple GAO reports have identified the potential 
value that systems engineering can provide to the technical stability, cost stability and 


positive schedule performance of a DoD acquisition program. (GAO, 2010). 


E. THE BLAME GAME 
1. Issues Often Blamed on Systems Engineering 


There can be cultural, financial, educational, structural, and political barriers to 
understanding the problems and implementation of possible solutions. People are 
comfortable in their own skill set and operate within that ability, sometimes to the 
detriment of what is actually required. PMs function in their acquisition role, just as 
engineers function more comfortably in their technical arena. To force a PM to function 
as an engineer, and vice versa, provides great personal learning opportunities, but can 
also expose a program to greater risk as a function of the learning process that occurs 


when a person is placed into a new position. 


The underlying trigger that creates the complex interdependencies in technology 
and systems engineering was incorrectly identified by SoSE, RAND, Director, Defense 
Research and Engineering (DDR&E) and other engineering organizations as a catch-all 


fix. 


Differences in perception of systems engineering vary considerably from 
organization to organization; a problem that is exacerbated by the Army’s stovepipe 
organizational structure. Some structural and political barriers exist with good intention- 
that intention being the sheltering of ways that work well for the uniqueness of the Army. 
There may be resistance to good ideas that work elsewhere, but that are not viewed as 
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adaptable to “the Army way.” These and other types of good intentions, such as service 
loyalties and pride of ownership, can have second- and third-order effects including lack 
of jointness among systems, competing initiatives, and support issues that are 


counterproductive. In this thesis, we seek to expose these counterproductive issues. 


Ze Determining the Root Cause of Failure 


Based on conversations with the SoSE, it is believed that the Army needs senior 
systems engineers to do adaptive engineering and programmatic system of systems 
integration. As a starting point, systems engineering in NASA was heavily and 
classically engineering-centered. NASA is risk-adverse, methodical, and prone to relying 
heavily on modeling and simulation before execution. The U.S. Navy is classically 
trained with emphasis on ensuring successful programs through rigorous academic 
instruction. In contrast to these organizations, the Army takes risks in program 
execution, as evidenced by programs such as FCS, Crusader, and System of Systems 
Common Operating Environment (SOSCOE)*. Educational institutions, although able to 
teach engineering, have limited ability to impart the tactical experience that may be 
necessary to build into the end-state system/weapon/unit capability the flexibility that the 


Army and all services need. 


The SoSE perspective must still acknowledge that stakeholders with different 


points of view will evaluate priorities differently. 


e Who are these stakeholders? 

e What is their point of view, and how does that influence their opinion of 
the value/role of systems engineering? 

e Who has the ability to operate cross-PM and cross-PEO (if not the systems 
engineers)? 


To expect all capabilities to be resident in a single individual is unrealistic and 
unproductive because a single person cannot be expected to be a certified expert in all of 


the above-mentioned areas and still be a functionally productive employee. 


4 Tt should be noted that these programs were not high risk due to technology related issues. Instead, 
these programs were deemed high risk due to poor architecting design, poor integration, and poor execution 
of a poor architecture design. 


20 


Considering the importance that ASA(ALT) SoSE has placed on recruiting 
systems engineers, it is worthy to note that there is no OPM general schedule job 
classification for a systems engineer. At the start of this research project we considered 
this to be potentially an error. But a solid training and certification structure exists within 
the DoD to enable the correct placement of applicants into systems engineering positions. 
What remains to be done is to implement hiring guidelines to encourage use of these 


credentials as discriminators. 


Figure 3 shows in simplistic form the career path progression of an engineer or 
acquisition professional along the x and y axis. “Pure” engineering would progress from 
left to right along the bottom. PMs rise along a path along on the left side of the figure. 
For systems engineers to fulfill every expectation within both the engineering and 
acquisition communities, it is necessary to have all of the underlying requirements of 
both professions. However desirable, this is unrealistic and identifies why solutions 
within the PM’s program are best generated by teams. Without disagreeing with the 
analysis that the DoD needs more engineers and, in particular, systems engineers, does 
the Army need only systems engineers? Or, is something else needed to augment 


systems engineering? 
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Figure 3. | Career Path Progression of Engineer or Acquisition Professional 


Systems engineering at the undergraduate level can be found at selected schools, 
but systems engineering courses are more readily available at the graduate level of study. 
One factor that continues to drive academics toward graduate rather than undergraduate 
teaching of systems engineering is that, fundamentally, systems engineering is the 
integration of multipleS disciplines with the goals of meeting the user’s needs. 
Understanding and implementing best practices can more easily be accomplished by 
engineers with more experience. Using Figure 3 as a guide, increasing engineering 
knowledge, and systems engineering expertise in particular, leaves voids of knowledge 
between engineering and acquisition. Cross training between systems engineering and 
acquisition career fields would address this as a two-dimensional solution. Adding 
requirements analysis and generation that is accomplished by the Training and Doctrine 
Command (TRADOC), an activity that precedes development, creates a third dimension 


of depth not shown in this diagram. Although the 2-D model shown in Figure 3 is 


5 Some of which include Operations, Cost and Schedule, Performance, Training and Support, Test and 
Evaluation, Disposal and Manufacturing. 
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adequate to represent internal to the program expertise, the third dimension is useful in 
visualizing the program’s relationship to the Army’s requirements generation located 
within TRADOC. Although this graphical analysis is far from all encompassing, systems 


engineering alone is unlikely to be the sole cause of acquisition failures. 
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IV. CURRENT STATE OF SYSTEMS ENGINEERING 


A. WHY SYSTEMS ENGINEERING IS IMPORTANT 


Systems engineering is a specialty that has been gaining ground since the late 
1940s; however, the DoD did not officially begin applying systems engineering practices 
until 2009. The actual ground gained is still minimal compared to the overall field of 
engineering. According to the National Science Board’s (2010) “Science and 
Engineering Indicators” report, a total, of 68,227 undergraduate engineering degrees were 
awarded in 2006. By comparison, only 723° engineering degrees were awarded in the 
field of systems engineering during the 2006 calendar year (Engineering Manpower 
Commission, 2006). Training in the field of systems engineering has been incorporated 
into the Systems Planning, Research, Development, and Engineering (SPRDE) career 
field by the DAU. However, the implementation of systems engineering practices by 
non-systems engineers is still widely prevalent in the DoD due to the inconsistent 
utilization of trained systems engineers. Anecdotal evidence based on multiple 
conversations within DoD acquisition communities have led us to infer that many 
systems engineering positions are filled by non-engineer managers who do not hold 
engineering degrees. While managers are capable of systems thinking’ this is usually 
applied to non-engineering work, which does not require the same level of rigor applied 
to a systems thought process as systems engineering requires (Franks & Waks, 2004). 
This creates a disparate level of understanding and functional capability between junior 
personnel who are expected to understand and perform systems engineering functions, 
senior staff members who may be classically trained in systems engineering, and those 
who have “become” systems engineers simply because the signs on their office doors 


label them as such. 





© Included in the total 68.227 as identified by the National Science Board’s 2010 report. This number 
does not include any graduates from DoD sponsored educational facilities. 


7 Systems thinking allows people to apply their understanding of social based systems explicit and 
improve them in a similar way that engineers use engineering principles to make explicit their 
understanding of engineering systems. 
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The European Space Agency describes systems engineering as what turns “an 
initial idea into a full system description, with all necessary elements integrated into a 
complete whole.” They further state that: 

Systems engineers maintain the focus on the space system as a whole 

rather than a collection of functional elements through regular project 

reviews occurring during subsequent 'Phase C/D' development, production 

and testing. These serve to ensure the mission remains on track. Systems 


engineering also guides technology development and assesses the impact 
of new technologies. (Why is Systems Engineering Important, 2009) 


Many organizations have postulated that good systems engineering efforts early in 
the life of a program will result in improved schedules, lower cost, and better product. 
NASA conducted a study to analyze it. In the late 1980s, Werner Gruhl of the NASA 
Comptroller’s office set out to improve cost estimation on NASA projects. As part of his 
effort, he mandated that NASA projects track costs to a common Work Breakdown 
Structure (WBS) that would allow gathering data across projects. This additional 
tracking was funded as part of each project. Over several years of live and historic 
projects, he developed the chart shown in Figure 4 that shows the impact of front-end 
investment (i.e. system definition and analysis) on the accuracy of cost estimation (Gruhl, 


2003). 
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NASA Tracking 1980s 


Impact of “Front-End” Investment 
Pre-Cost Commitment Investment vs. Total Cost Growth 





Total Program Overrun 
32 NASA Programs 














LANDSAT.D 





wi 
= 
Ez 
b 
ow 
oO 
er 
oO 
ref 
Or 
Sw 
“> 
ro) 














: °o Investment in Source Wemer Gruhl 
System Engineering Effort(Si 5) NASA Comptroller's Office 





Figure 4. Impact of “Front End” Investment (From Gruhl, 2003) 


Despite the noted wide dispersion of data, NASA contends that this provides 
ample evidence for systems engineering investment. In this particular study, the findings 
were used to recommend a 10-15% investment of program funds to the effort. However, 
the study does caution that poor quality systems engineering reduces the effectiveness of 


any potential gain. 


This assessment was reinforced during a 2004 presentation to the 14th Annual 
International INCOSE Symposium in Toulouse, France where Mr. Eric Honour presented 
a Statistically relevant study which concluded that increasing the level and quality of 
systems engineering has a positive effect on cost compliance, schedule compliance and 


subjective quality of the projects (Honour, 2004). 


There have been multiple studies performed since 2000 that have described the 


need for a robust systems engineering capability, but none make a more compelling 
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argument than a 2008 report published by the Air Force Studies Board that studied 


multiple USAF acquisition programs and came to some common findings. This report 


made the following statements: 


There is a need for an appropriate level of systems engineering talent and 
leadership early in the program, with clear lines of accountability and 
authority. Senior systems engineering personnel should be experienced in 
the product(s) domain, with strong skills in architecture development, 
requirement management, analysis, modeling and simulation, affordability 
analysis, and _ specialty engineering disciplines (e.g., reliability, 
maintainability, survivability, system security, and technology maturity 
management). (AFSB, 2008, p. 49) 


There is a need to establish and nurture a collaborative user/acquirer/ 
industry team pre-Milestone A to perform system trade-offs and manage 
overall system complexity. Today, there are often significant disconnects 
in the hand-offs between users, acquirers, requirements developers, 
industry, and others. Some of the “best practices” include structured 
collaboration among these members. (AFSB, 2008, p. 50) 


One must clearly establish a complete and stable set of system-level 
requirements and products at Milestone A. While requirements creep is a 
real problem that must be addressed, some degree of requirements 
flexibility is also necessary as lessons involving feasibility and practicality 
are learned, insights are gained, technology is matured, and the 
development subsequently proceeds. Certainly control is necessary, but 
not an absolute freeze. Also, planning ahead for most likely change 
possibilities through architectural choices should be encouraged, but 
deliberately managed, which is a concept encouraged herein. A typical 
program execution team has a program manager (PM)-level systems 
engineering integration team (SEIT), with responsibility, authority, and 
accountability to perform the systems engineering functions (including 
analysis, modeling and _ simulation, architecture development, 
requirements management, and so on). Some of the “program discipline” 
needs to be in pre-Milestone A management. (AFSB, 2008, p. 50) 


It is necessary to manage the maturity of technologies prior to Milestone B 
and to avoid reliance on immature technologies. Technology maturity and 
risk mitigation plans should be carefully managed as an integral part of 
program plans. (AFSB, 2008, p. 51) 


The above statements represent findings from the USAF study as a result of 


successes and failures that were achieved during USAF acquisition programs. These 


results serve as guideposts to successful product and program development and are 


applicable to DoD and U.S. government acquisition programs in general. While this 
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report did not directly result in any new policies being enacted, it is our belief that the 
commissioning of this report by the AFSB is indicative of the importance that the USAF 


places in systems engineering. 


Although the SoSE is reading reports obtained from the office of the DDR&E 
(Welby, 2010) and having discussions with the Army Acquisition Executive, both of 
which identify systems engineering as the root cause of program failure, the list of must- 


have improvements identify engineering as only one component of the needed fix. 


Program failure is a combination of interrelated problems. We identified one 
problem causing failures of programs to be personnel in systems engineering positions 
with training less than 100% complete. This is linked with the complexity of the 
technological aspect of the program as a system and its place in the system of systems. In 
a sense, people in these positions were in over their head. Another portion of the problem 
falls within the realm of an acquisition professional rather than in systems engineering. 
The final portion is organizational lack of commitment that PMs and PEOs have to train 


their staffs. 


B. WORKFORCE STATUS 


According to DDR&E, the DoD is lacking in DAU certified systems engineers 
(Welby, 2010). Since the Army is subordinate to the DoD, and their certification 
numbers are included in the report from the office of DDR&E, one can infer that the 


Army is similarly lacking DAU certified systems engineers. 


Clearly, training and certification is available to the DoD with a recognized level 
of standardization from a variety of sources. But this has not “fixed” the Army’s dearth 
of systems engineering expertise. The problem may be structural inhibitors that prevent 
student attendance and/or a perception of too narrow of an acquisition focus for the 
research and development (R&D) or test and evaluation (T&E) communities. INCOSE 
described certification in this way: “Certification is a formal process whereby a 
community of knowledgeable, experienced, and skilled representatives within an 


organization, such as INCOSE, provides formal recognition that a person has achieved 
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competency in specific areas (demonstrated by education, experience, and knowledge)” 
(INCOSE, 2010b, para 2). No current certification numbers for the Army or the DoD in 
general are publicly available for INCOSE certifications. 


In a January 19, 2010 briefing to the 6th North Atlantic Treaty Organization 
(NATO) Life Cycle Management Conference in Brussels, Belgium, Mr. Nicholas Torelli 
(2010a), Director of Mission Assurance, Systems Engineering Directorate, Office of the 
Director of Defense Research and Engineering, United States DoD, provided data that 
showed definitively that the U.S. DoD acquisition workforce is largely comprised of 
personnel with more than 25 years of service. During this same briefing, Mr. Torelli 
concluded that the majority of the current DoD acquisition workforce has entered the 
portion of their career during which they should be mentoring and training the incoming 
workforce. Mr. Torelli noted that the incoming workforce is sorely lacking in practical 
experience in the field of systems engineering, and explained that one of his 


organizational challenges is to ensure that the USD(AT&L) is able to: 


e Better manage workforce development requirements and certification 
standards 
° Make better decisions about human capital strategy and initiatives for the 


systems engineering workforce 


e Provide acquisition programs with the quantity and quality of systems 
engineers that they need to be successful 


e Enable USD(AT&L) to better determine shortfalls at all levels in both 
competencies and workforce size 


Briefings held since late 2008 in the systems engineering arena (Jaggers, 2010; 
Shaper, 2008; Torelli, 2008; Vannucci, 2008, 2009; Vannucci & Barnabe, 2008; Welby, 
2009, 2010) have echoed one common DoD overarching goal: “[to] develop future 
technical leaders across the acquisition enterprise” (Welby, 2010). Each of the 
presentations that echoed this goal has noted that the actual execution of the goal is 


extremely difficult. 


A conspicuous example of improper personnel placement is the finding that, in 
some instances, the systems engineer is a systems engineer in name only. On projects 


personally familiar to the authors were systems engineering billets filled by persons with 
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no systems engineering training, and in some cases, no engineering training at all. Blame 
in a situation like this would fall on the systems engineering community if the program 


fails, but it is actually a failure of the personnel selection process. 


In contrast, an excellent example of why effective systems engineering is a 
valuable goal is the recent success that the Mine Resistant Ambush Protected (MRAP) 
program has experienced using systems engineering best practices during budget drills 
for life cycle management. Kevin Fahey, Program Executive Officer Combat Support 
and Combat Service Support, is quoted as saying “applying systems engineering best 
practices and Lean Six Sigma (LSS) principles to the MRAP requirements management 
process enabled the JPO (Joint Program Office) MRAP to reduce process inefficiencies, 


providing an unprecedented cost avoidance to DoD” (Osborn, 2010). 


C. TRADITIONAL SYSTEMS ENGINEERING FUNCTIONS DONE 
OUTSIDE THE ACQUISITION ARENA 


TRADOC serves as the user’s representative to establish what the product must 
do or performance specifications, commonly referred to as requirements. Requirements 
would also include the condition under which the performance should be expected. For 
example, the performance expected of a battery in desert versus arctic climates might be 
different. Some engineering skills are needed to ensure that the specification handed to 
PMs is either within the realm of the possible (and affordable) or at least worthy of 
research and development to make it so. TRADOC follows guidance from the acquisition 
community and defines performance specifications rather than identifies the material 
solutions. Does it matter that the requirements managers, specifying the performance of 
the product and identifying the context of that system within the system of systems, are 
not systems engineers, or engineers at all? The overlap between TRADOC’s efforts and 
the formal analysis of alternatives that systems engineers should actively participate in is 


significant. 


In Figure 5, the relationship between warfighters, TRADOC, and the material 


developers is a two-way flow with needs—specifications—and product in the outer circle 
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and feedback to improve product in the inner circle. The mere fact that this classic model 
is used more often than any other is indicative of the omission of other important 


organizational perspectives. 





Warfighter 






Material 


TRADOC Developer 











Figure 5. Classic Development Cycle 


Missing from Figure 5 is the Army’s Test and Evaluation Command (ATEC), and 
all of the elements of the U.S. Army Research, Development, and Engineering Command 
(RDECOM). Inclusion of these entities is shown in Figure 6. ATEC is needed because it 
is their evaluators that determine product maturity or suitability. Consultation on testable 
metrics would be advisable. RDECOM often is on the cutting edge of the dividing line 
between achievable and not feasible. There may be workload, interdisciplinary systems 
inexperience, or other limitations that make this less than idea. However, personnel 
transfers between TRADOC, RDECOM, ATEC, and the Material Developer are not fluid 
and this limits potential cross-pollination benefits. The benefits of systems engineering 
personnel transfers amongst these organizations includes, but is not limited to, a shared 
outlook that creates a greater holistic universal perspective for analysis of alternatives, 


requirements generation and selection of evaluation criterion. 
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Figure 6. “Modified” Development Cycle 


D. ORGANIZATIONAL AND CULTURAL ISSUES 


In the life of a program, systems engineering is critical in the early stages. It is 
inconceivable that a PM would hire or promote his systems engineer into the program’s 
staff just in time to send him off to training. It is natural for leadership to want to hold on 
to their critical personnel and release non-essential personnel for school. What happens 
when that key individual cannot or will not go? In effect, training may be offered to 
those less likely to be the best. Competition for PEO-managed training dollars may also 
be an inhibitor to employee access to training. Depending on the fiscal health of a PEO, 
training opportunities may be limited. These structural barriers exist because of the 
environment in which PMs operate. Most PMs will want their systems engineers trained 


and certified, but will expect it to be done on someone else’s time and budget. 


Only senior management at PEO and above can institute a change in the culture 
that rewards not only those who manage to take training, but also those that sacrifice so 
that training can be done. We postulate that lack of familiarity with what the DAU, NPS, 


and other dedicated systems engineering postgraduate institutions can offer is attributable 


a3 


in part to apathy. Many personnel do not seek training if it is not required. Leadership 
does not require it because they do not want to pay for it, or excuse personnel to attend 


training. 


Transforming the workforce will require a different mentality, a new paradigm 
that rewards individuals for their initiative in seeking and taking training, encourages a 
leader to let subordinates get the certification, and possibly requires completion within a 
set time to earn credentials from initial entry into a systems engineering position. 
Perhaps linking the pay increase of promotions to successful credentials would provide 


enough incentive. 


It is also useful to note that in larger systems engineering organizations like 
ATEC and RDECOM, senior personnel would also be working toward their own 
certification and may be somewhat more sympathetic to subordinate requests for training. 
This cooperative attitude may be further incentivized by encouraging cross- 
organizational transfers from acquisition organizations, such as PMs or PEOs, or 
TRADOC locations to ATEC and RDECOM to enable training and to further increase 
interdepartmental coordination. By making budgetary allowances to organizations that 
are better able to facilitate this type of training, personnel can rotate through those 


commands and then return to their sponsor organizations. 
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v. FINDINGS 


A. RECRUITMENT 


OPM estimated in 2009 that 57% of full-time, permanent federal employees, as of 
October 1, 2006, would be eligible to retire by 2015 (OPM, 2008). This may place some 
departments at risk of a “brain drain” if too many experienced workers and managers 
leave at once. At the same time, however, it also presents an opportunity to bring new 


talent into the workforce to build a solid foundation for the future. 


It would be a misperception for the Army to believe that merely increasing the 
number of systems engineering in the acquisition community would satisfy systems 
engineering recruitment objectives. Quantity must be balanced with quality. While 
quantity goals can be determined for open position numbers and attrition rates, quality 
goals will be more subjective. These goals could include: degree type (since few will 
have an undergraduate degree in systems engineering), grade point average (GPA), the 
ranking of the school, prior certifications, and prior work or experience factors. Prior 
certifications include, but are not limited to, certification as a Certified Systems 
Engineering Professional under INCOSE’s certification process, or one of the 
certification levels within DAU that are associated with the Systems Planning, Research, 
Development and Engineering-Systems Engineering (SPRDE-SE), SPRDE-Program 
Systems Engineering (SPRDE-PSE), or SPRDE-Science and Technology Management 
(SPRDE-S&TM) fields. Desired quantity and quality can then lead to successful 


recruiting that refills the ranks of the Army’s aging engineering workforce. 


Recruitment is not an event; it is a process. Moira Hanna (2010), explained 
recruitment as being comprised of several steps: “applicant generation, maintaining 
applicant status, and applicant job choice/decision.” After deciding on which skills to 
add to the workforce, and which is a preparation phase preceding recruitment, the 
government must determine both a method of reaching out to potential applicants, and 


where to direct efforts (Hanna, 2010, p. 1). 
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The challenges in recruiting are great when an agency is working against current 


undergraduate student thought processes, as found in a recently published 2009 survey by 


the Partnership and Universum USA group (Partnership and Universum USA, 2009). 


This survey resulted in the following findings, as shown in Figures 7 and 8: 


GOVERNMENT/PUBLIC SERVICE AS AN IDEAL INDUSTRY BY MAJOR AREA OF STUDY 


Interest in government service is lower among individuals in groups that 
the government needs _ most. For example, students with 
technical/scientific majors are less interested in government and public 
service than non-technical majors from similar universities government 
policy. 








The Universum IDEAL™ Employer Survey 2008, Undergraduate Edition, American Students 


Liberal arts 34% 


Natural sdences 16% 


IT 13% 
Business 10% 


Engineering 9% 








Figure 7. 


Government/Public Service as an Ideal Industry by Major Area of Study 
(From Partnership and Universum, USA, 2009) 


Salary expectations are high. Respondents® expected to earn an annual 
salary of more than $49,000 in their first job after graduation. In contrast, 
starting salaries for entry-level federal government employees with 
undergraduate degrees typically range from about $30,000 to $38,000, 
adjusted by locality. 


8 Respondents are from a pool of 31,876 undergraduate students at U.S. universities who participated 
in the Universum USA’s 2008 annual survey. 
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REMUNERATION AND ADVANCEMENT OPPORTUNITIES AS AN ATTRIBUTE OF EMPLOYER ATTRACTIVENESS 
INTERESTED IN GOVERNMENT/PUBLIC SERVICE ® COMPARED TO ALi AMERICAN STUDENTS @ 
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Overtime pay 10% 


Other 0% 











Figure 8. | Remuneration and Advancement Opportunities as an Attribute of Employer 
Attractiveness (From Partnership and Universum, USA, 2009) 


1, Applicant Generation 


The military arm of the DoD is more rooted in methods and in the infrastructure 
to recruit than its civilian counterpart. The Reserve Officer Training Corps (ROTC) on 
college campuses is conducted with awareness of, and in cooperation with, local 
recruiting offices. Although it is a separate chain of command and operating under 
different quota systems, the ROTC has an established presence that is immediately 
recognizable and updated, and that operates within the digital vernacular of web pages 
and social media used by the men and women they want to meet. It is beyond the scope 
of this paper to determine whether the ROTC and Army Recruiting Command 
infrastructure can be leveraged for Army engineering recruitment, but it is not unrealistic 
to consider reserve commissioning paired with civilian government service after 


graduation. Currently, there is a program offered by the Naval Surface Warfare Center, 


Panama City, Florida that could be leveraged by the Army, but further analysis is 
necessary to determine if this program would support the needs of the Army. Some 


defense industry partners are using similar recruiting tactics. 


Boeing Aerospace has been using social media such as Facebook as early as 2007 
for advertising, contests, and giveaways (Chang, 2007). With nationwide access via the 
Internet, the Army can target interns, as well as future workforce. Internships often lead 
to new hires that have a better base understanding of the job they are hired to do. One 
benefit of internships is the recruitment effort conducted by the intern after he returns to 
campus to complete his schooling. These are the types of social media tools the Army 


needs to promote hiring. 


2: Combating Financial Misperception 


Economic forces have made government careers more desirable during the 
economic downturn of 2009-2010. Salary expectations are traded for job security. That 
incentive will not be as dominant of a factor after the economy recovers. What can the 
government offer instead? The government’s ability to bring engineers onboard who lack 
experience and offer follow-on engineering or acquisition training, gives prospective 
newcomers more to consider. Certification or educational assistance outside of core 
engineering could also be offered. Army recruiters often use educational opportunities to 
entice people to join the service. Why not apply the same logic to postgraduate education 
for those that merit the benefit? The Army also offers student loan forgiveness to 
soldiers with undergraduate degrees. For select specialty skills, loan forgiveness would 
be worthwhile for the Army in order to fill key positions. Less desirable jobs, hard-to-fill 


vacancies, or assignments to hardship locations can be tied to greater benefit packages. 


It is a commonly held misperception that defense contractors are typically paid 
more than their government counterparts. This perception appears to be misplaced. 
Figure 9 shows a snapshot taken from the salary review site Glassdoor.com, which shows 


that the average? salary of a systems engineer is around $82,000 per year 


9 Salary data was taken from a random sample of 702 salaries based on the Systems Engineer job title, 
from the salary information website Glassdoor.com. 
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(Glassdoor.com, 2011a). Finding a government salary comparison is difficult, because 
the lack of specific salary reporting. Glassdoor.com has a smaller data set of salaries that 
average!9 out to $77,600 per year (Glassdoor.com, 2011b). This snapshot of government 
salaries, for all engineering position data available, is reflected in Figure 10. Even with 
this data showing a five percent difference in salaries, recruiters trying to fill positions 
that offer lower pay have to use other incentives to combat the commonly held 


misperception in order to attract applicants. 


10 Salary data was taken from a random sample of 21 salaries based on the Engineer job title from the 
salary information website Glassdoor.com. 
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Figure 9. 


Snapshot of Randomly Selected Data on Available Salaries for Systems 


Engineering Jobs Within the Private Sector (From Glassdoor.com, 2011) 
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Figure 10. Snapshot of All Available Engineering Salary Data (From Glassdoor.com, 
2011) 


3. Applicant Quantity and Quality 


A larger pool of interested potential hires is one method of ensuring that enough 
applicants are able to meet the needed qualifications. Too many organizations claim to 
be hiring “the best and the brightest” without qualifying their use of that phase. David 
Halberstam (1972) coined that phase for the title of his book, which describes the John F. 
Kennedy Presidential team mired in Vietnam, in order to capture a sardonic, rather than 
flattering tone (Rich, 2008). But the real need for systems engineers is unlikely to be met 
by only new graduates, however academically ranked. This is because, as we noted 
earlier, experience is essential for adaptive engineering within the context of what the 
Army wants to accomplish with system of systems engineering and integration. 
However, even advocates of the “grow your own” engineering force will admit that a 


substantial base is necessary as a starting point. 


Recruiting is important, but as The Honorable Mr. Ashton B. Carter, 
USD(AT&L), stated in his 2010 interview with Defense AT&L magazine, “workforce 
size is important, but quality is paramount” (Anderson, 2010, p.7). The key to ensuring 
that quality recruits are found across all levels of the acquisition field is to ensure that the 
recruitment begins before the current senior level of governmental employees start 


4] 


retiring. It may take time to find quality and it may even be necessary to grow more 


experienced quality from within if it cannot be found elsewhere. 


The easiest way to ensure that recruiting begins quickly, is to leverage internships 
and other entry-level intern programs, which will allow the government to flexibly recruit 
personnel and provide on-the-job training (OJT). In this manner, classroom learning is 
supplemented, and candidates experience OJT real-world scenarios, in order to determine 


if each candidate is correct for the position, or can be helped to grow into it. 


4. Recruiting Practices 


Employers must seek access to new ideas and viewpoints by expanding the 
current search for new middle-level talent from outside the profession - that is, to search 
for more than traditional engineering graduates. They must recruit from other technical 
fields such as information technology (IT), physics, chemistry, and biology. This can be 
summarized by simply stating that one must consider resumes that do not look like the 


resume of the hiring official. 


A mistake made in current student recruitment is to underestimate students’ 
knowledge and abilities—that is, to “pitch” too low. Students today are often better 
educated in specific technical subjects than their teachers (Partnership for Public Service, 
2010). There has been much progress in school curricula in recent years, but because 
education systems tend to sustain and replicate themselves, major changes are often 


rejected regardless of their merit. 


The following guidelines will enhance the motivation, education, and training of 
young people: 


e Establish and maintain contact with young people throughout their 
education and their transition into the ranks of employees. 


e Make contact not solely with students, but with all those who impact their 
decision-making: parents, teachers, student advisers, career guidance 
counselors, school administrations, among others. 


e Establish and encourage partnerships among professional engineering 
associations, colleges, industry, and federal, state, and local government 
agencies. support scholarships and internships, and 
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e Provide hands-on student research opportunities such as access to 
government acquisition programs. 


Other government agencies are already participating in these sorts of internship 
recruitment efforts. For example, many of NASA’s external hires for entry-level 
positions have been through the Cooperative Education Program, which provides NASA 
centers with the opportunity to develop and train future employees and to assess the 


abilities of potential employees before making them permanent job offers (GAO, 2008a). 


Fortunately, mechanisms are already in place for agencies to capitalize on 
successful internships by hiring students. The federal Student Temporary Employment 
Program (STEP) and the Student Career Experience Program (SCEP) not only provide 
work experience that directly relates to a student’s academic program and career goals, 
but also SCEP allows for noncompetitive conversion to term, career or career-conditional 


appointments. 


B. TRAINING AND CERTIFICATION 


Figure 11 serves as a guide for understanding the education progression for Army 
engineers and acquisition experts. The engineer in an acquisition support role learns 
more about aspects beyond their initial specialty and ideally would follow a path to 
systems engineering. This is different from continuing in a specialized engineering 
education that would maintain movement on the horizontal line. Learning DoD 
acquisition and systems engineering is not likened to a master’s degree in mechanical 
engineering. This is because the systems engineering taught in DAU would be focused 
on the way the engineer supports the PM. The hypothetical jack of all trades resides at 
the pinnacle in the upper right corner, which we have labeled Inter-PEO Systems 


Integrator, because the skills are neither solely engineering, nor programmatic. 
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Career Path 
(An Army Example} 


Acquisition Intra-PEO Inter-PEO Inter-PEO 
Expert Collaboration Collaboration Systems 
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Here there be * ” 
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Dragons... System of 
Systems 
—— 
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Engineering 





Engineering Skills and Knowledge 


Figure 11. Career Path Progression for Systems Integrator 


Systems engineering for a typical PM in this circumstance is subordinate to 
system of systems engineering or systems integration. The systems engineer looks 
inward over the domain of the PM or PEO. The system of systems engineer looks 
up/outward at the next levels in the hierarchy and laterally amongst peer programs to 


determine how their respective efforts can combine to fit together as a whole. 


At the lowest levels, exceedingly specialized knowledge in a particular area is 
needed. Development expertise overshadows integration expertise. But at each 
successive step, the realm of an integrator involves increasingly broader skills over 


multiple areas. 


Referring again to Figure 11, as an acquisition professional increases his scope, he 
becomes an inter-disciplinary integration expert who is able to keep contributing PMs 


and their programs aligned. Engineering is only one of those disciplines. As stated 
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before, the chart has “Integrator” in the upper right corner rather than “Engineer” for a 
reason. The Master Integrator may or may not have the title of engineer, but she will 
have engineering training. Likewise, the Master Integrator may not have held an 
acquisition position as a PM, but she will have taken the training. ASA(ALT) expert, Jon 
Englebrektson (2010), coined the position as a “Program of Programs Manager” and a 


partner to the system of systems engineer. 


INCOSE has also created a multilevel certification program (INCOSE, 2010b). 
This program recognizes the skills of a variety of enrollees and certifies them at various 
stages in their career. While this may be a clearly recognized and very portable 
certification, it may not be easily worked into the busy schedule of the Army civilian. 
INCOSE certification levels are depicted in Figure 12. The ability to add extensions to 
the certifications, such as a specialty in acquisition, is illustrated in the right-hand side of 


the figure. 





Senior Level 







Foundation Level 
Extensions 


Entry Level 


Expert Systems Engineering Professional 


Certified Systems Engineering Professional 
Associate Systems Engineering Professional 
US DoD Acquisition Extension 











Figure 12. INCOSE Certification Program Progression (From INCOSE, 2010b) 


45 


NASA has developed a similar approach, as shown in Figure 13. Core 
competencies overlap between project management and systems engineering. However, 
the NASA structure and approach, as noted before, does not “fit” perfectly in the Army 
(NASA, 2010). 














Figure 13. NASA Project Management and Systems Engineering Competency 
Framework 


The key variable, however, is building greater awareness for the field of systems 
engineering and ensuring that the right kinds of skills are being applied toward these 
positions. Solving that important challenge could go a long way in helping overcome 
society’s technology challenges and creating a skilled workforce that can more readily 


find valuable employment opportunities (Amaba, 2010). 


The number of college and universities offering programs in systems engineering 
is increasing as students recognize the employment opportunities available in both 
government and industry. Schools with smaller systems engineering programs are 
expanding them as the rate of interest increases (Amaba, 2010). With academia course 
material currently in an evolutionary stage, how can the Army ensure standardization of 
the systems engineering educational levels of the applicants it receives who have degrees 
in systems engineering? The DAU, available to all DoD employees to train in a variety 


of career fields, is a source for possible standardization. 
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In response to the perceived need for systems engineers and systems engineering 
training, the DAU has developed a three-level training and certification program for 
systems engineers and program systems engineers (see Appendices B and C). These 
programs allow for a wide range of participants and skill levels, from the newly hired to 
the more experienced personnel. Experienced personnel are described by the DAU 
(2010a) as individuals who have four years of technical experience in an acquisition 
position. Of that experience, at least three years must come from positions in SPRDE- 
SE, SPRDE-PSE, or SPRDE-S&TM. The remaining year of experience may come from 
positions in IT, Test and Evaluation, Production Quality Management, PM, or Life Cycle 


Logistics. 


Similar experience gained from other government positions or industry is 
acceptable as long as it meets similar standards. Experience is further broken down into 


type of assignment. These are categorized as: 


e Functional specialist 

e Software/IT engineer 

e Developmental engineer 

e Science and technology research engineer or scientist 


Relatively clear definitions of associated duties can be found in the DAU 
Certification Guide for each of these assignments at each of the three levels (DAU, 
2010a). Completion of course modules for each level of DAU SE certification, per the 


DAU SE Certification Guide, ensures some standardization of quality and competency. 


Core Certification Standards are published as guidelines for acquisition, 
functional training, education, and experience. DAU courses available in the “Core Plus 
Development Guide” (DAU, 2010a) are clearly listed and broken down for each 
assignment type. As a side benefit, this certification structure addresses training for 
systems engineers operating in traditional engineering roles and the positions of 
Integrator or Program of Programs Manager (J. Engelbrektson, personal communication, 
December 13, 2010). Clearly, the perceived need for training from the context of an 


acquisition professional can be readily fulfilled. 
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On larger programs, with program elements co-located, an alternative training 
option is to bring the trainers to the program area and have the training conducted on-site 
with the project. The trainers come to the project and “educate” the systems engineers on 
exactly what they need to do and the next steps to take. The FCS, as an Army example, 
was spread across multiple states and is a program that would not have lent itself to this 


training solution. 


C. RETENTION 


The loss of experienced employees, due to retirement or to more promising 
opportunities, can deal a serious blow to an agency’s operational capacity and 
performance, if the departing workers leave with institutional knowledge and 
organizational savvy that up-and-coming staffers do not yet have. Attrition and retention 


are important indicators about the state of the workplace environment. 


Any job (even within the government) must offer a rewarding lifestyle. Managers 
and supervisors of government civilians should seek employees’ guidance on their work 
environment and recognize that especially with today’s young people, flexibility and the 
use of the most current electronic tools are of importance. Retention can be as simple as 
ensuring that employees are being used to their fullest possible capabilities. The 2008 
report by the Merit Systems Protection Board to the President of the United States found 
that employees overwhelmingly agreed (91%) that their work was important, while one 
third (32%) indicated that their job did not make good use of their skills and abilities 
(U.S. Merit Systems Protection Board, 2008). 


Another key element in retention is creating a revised attitude toward failures: 
instead of chastising those whose ideas or projects do not succeed, employers must now 
recognize the value of failures as a way to learn not only how to prevent future failures, 
but also how to open new pathways to successful results. More and more employers have 
begun to tolerate failures by their youngest engineers and provide them with the 


resources needed to assure greater successes later (AIAA, 2009). 
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Employees should be encouraged to develop project management skills, and be 
given the opportunity to learn a broad spectrum of jobs rather than focusing on a single 
one. They should receive recognition of their ability and their contributions to society 
and the profession. As stated earlier in this chapter, training is available and employees 
that are allowed access to that training are more likely to stay with their organizations. It 


is up to employers to make it happen. 


Employers should not foster “workaholics” by setting the example of 24/7 work; 
instead, they should encourage a life outside the work place, and they should strive to 
work a 40-hour week. All workers, regardless of position, should be given at least a 
summary of the key points of the company strategy. Typically, about two thirds of 
employees do not involve themselves in their company’s goals, and nearly half are totally 
disconnected from their employer (AIAA, 2009). Employers should also ensure that 
employees understand their role in the greater good and that the employees make a 
different in the lives of other people (AIAA, 2009). These two ideas are reinforced by 


the survey data summarized in Figure 14. 





CAREER GOALS OF AMERICAN STUDENTS INTERESTED IN 
SELECTED INDUSTRIES FOR EMPLOYMENT AFTER GRADUATION 






















Students Interested in 
Nonprofit/Not-for-Profit Industry 


All American Students Interested in 


Career Goals Students Government/Public service 





To be a leader or manager of people 
To be a technical or functional expert 15% 12% 


To be autonomous or independent 14% 13% 





To be competitively or intellectually challenged 


To be entrepreneurial or creative/innovative 


To be secure or stable in my job 46% 44% 





To have an international career 17% 





To have work/life balance 


The Universum IDEAL” Employer Survey 2008, Undergraduate Edition, American Students 











Figure 14. Career Goals of American Students Interested in Selected Industries for 
Employment After Graduation (From AIAA, 2009) 
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According to a 2010 report published by the Partnership for Public Service and 
Booz Allen Hamilton (Partnership for Public Service & Booz Hamilton, 2010), retention 
can be best summarized, as depicted in Figure 15, by ensuring that a balance is met 


between the four major areas that describe needs that all employees have in order to feel 


valued and happy: 
e Teamwork, Supervision, and Leadership 
e Performance Management, Compensation, Benefits, and Work/Life; 


° Agency Mission and Employee Skills Match 
° Employee Development and Support 





Work environment framework 










AGENCY MISSION AND 
EMPLOYEE SKILLS MATCH 


TEAMWORK, SUPERVISION 
AND LEADERSHIP 


STRATEGIC 


LEADERSHIP PURPOSE 


TEAMWORK AND SKILLS AND 
SUPERVISION MISSION MATCH 


WORK 
ENVIRONMENT 


PERFORMANCE SOCIALIZATION AND 
MANAGEMENT SUPPORT FOR DIVERSITY 


WORKLIFE, PAY TRAINING AND 
AND BENEFITS DEVELOPMENT 


PERFORMANCE 
MANAGEMENT, 
COMPENSATION, 
BENEFITS AND WORK/LIFE 


EMPLOYEE DEVELOPMENT 
AND SUPPORT 








Figure 15. © Work Environment Framework (From Partnership for Public Service & Booz 
Allen Hamilton, 2010) 
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VI. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 


This joint applied project was created to answer the primary question: What 
recommendations does the ASA(ALT) SoSE need to make to the USD(AT&L) to ensure 
that the proper personnel are recruited, trained, certified, and retained, to increase the 
U.S. Army systems engineering capabilities needed to meet the increasingly complex 


requirements of the Army’s system of systems strategy? 


Over the course of researching and writing this joint applied project, we have 
come to conclusions that we did not originally expect. The technical aspects of training 
available to the systems engineering community within the DoD appears robust enough 
to provide value, but staffing the systems engineering community has been problematic at 
best. It is the implementation of proper recruiting, use of training, and retention (RTR), 
that have been the problem. A common theme across the U.S. government is that one 
rarely ever thinks that RTR is important until we hit a major crisis point, and then when 
things are slower no one is thinking about RTR because they are in the process of 


regrouping. 


RTR is a matter of leadership making RTR a priority for their people. It is a 
matter of supervisors and key management staff acknowledging that education and 


certification are important, more important than just getting the job done. 


If the Army acquisition community wants its people to augment and enhance their 
current ways of looking at problems and solutions, stay interested and focused, and retain 
its people and provide the necessary continuity that is required to support MDAPs, then it 
needs to create the proper work environment that allows the RTR actions to occur, 
without sacrificing the overall mission requirements. This appears to be applicable 
beyond the Army acquisition community; however, more follow-up study is necessary to 


determine true applicability. 
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To phrase it differently, a supervisor might say, “I can do without you for a 
couple of weeks, if it means you're coming back better and stronger than before.” What a 
supervisor saying that does to an employee, is leave the impression, “I'm valued by this 


organization and they're interested in my future.” 


Additionally, making certification one of the requirements for promotion, and for 
greater responsibility, helps to solidly convey an organizational leadership's commitment 
to their people. This way, the promotion requirements are codified in a manner in which 


people can readily understand where they are within the organizational structure. 


The key to a great organization has never solely been its ability to execute its 
technical mission as efficiently as possible. Leadership guru Warren Bennis best summed 
up this idea in the following quote: 

Good organizations make people feel that they're at the very heart of 

things, not at the periphery. Everyone feels that he or she makes a 

difference to the success of the organization. When that happens, people 


feel centered and that gives their work meaning. An organization is only 
as a good as its people. (Heathfield, 2011) 


B. RECOMMENDATIONS 


As a result of the data and analysis from this paper, we are providing the 
following recommendations for the ASA(ALT) in four categories: Overarching, 
Recruitment, Training & Certification, and Retention. These recommendations will 
consist of both recommendations and additional areas of focus that we believe the SoSE 


needs to consider as part of the process to fix their problem. 


Following are our recommendations for the SoSE. 


1. Overarching 

e Realization that changes to the systems engineering RTR process will not 
be a panacea for the problems that plague systems engineering for 
ASA(ALT) 

e Create an ability to articulate exactly what the Army is looking for from 


systems engineering personnel, to include: 


e Defining what activities a systems engineer is expected to perform 
in support of an acquisition program. 
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e Listing the artifacts of those performed activities. 
e Creating metrics to measure success. 


Ensure that recruiting, training and retaining employees is not a short-term 
goal and short-term fixes are not something that should be expected. 


Create incentives for systems engineering employees who want to stay in 
these positions. 


Develop a metric (or series of metrics) to ensure that the proper workforce 
size and quality are met. 


Develop a process that ensures that organic workforce growth is 
adequately met. 


Develop a system to ensure that proper retirement knowledge transfer 
occurs given the fact that 57% of the DoD acquisition workforce is 
expected to retire by 2015. 


Recruitment 


Establish an Army systems engineering recruitment strategy. 


Focus on creating a work environment that attracts personnel who would 
not normally be interested in government service. 


Increase focus on out-of-the-box candidates as the best candidate for the 
job, even if they might not appear to be the best one on paper. 


Improve the advertising to potential recruits in areas in which government 
service provides more value than the private sector (e.g., the ability to 
make a difference in real-world situations). 


Improve the ability of the recruitment process to include current DoD non- 
Army personnel into the overall recruitment process. 


Training and Certification 


Develop a rapport with education providers to present recommendations to 
influence the kind of curricula that are out there for systems engineering 
(e.g., more broad-based program management skills). 


Develop a cross training capability for new systems engineers coming into 
the government, such as a specialized systems engineer intern-type 
program so that these new graduates get a feel for the total acquisition 
process from the perspectives of different people, levels of responsibility, 
subject-matter experts, program offices, PEOs, etc. 


Focus on education and specialized experience to ensure that the right 
people are being selected for key positions. 
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Retention 


Ensure that the retention of good people is a focus for the leadership in the 
Army — cross-training opportunities, opportunities with industry, long- 
term training opportunities, and perhaps even a separate pay scale like 
there is for scientists needs to be an initiative that is a high priority for 
Army leadership. 


Develop a process that allows people who have the capability to be 
systems integrators to be recognized by management as able to take on 
systems engineering types of positions, even if they are not necessarily 
schooled engineers. Provide opportunities to attend conferences and 
symposium to allow for community recognition and involvement. 


Recognize personnel for their achievements in continuous learning. 
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A. 





APPENDIX A. RESEARCH MATRIX 


RESEARCH MATRIX DEVELOPED TO FOCUS RESEARCH 


with traditional engineering backgrounds and prerequisites 


What is a Systems Engineer Review of SE literature from course Incose definition 
What systems engineers need to know (handout) DoD definition (Def Acq Guidebook) 
Review of DDRE briefings Mis-identification instances 


Assessment of posititions currently identified as SE but NOT associated |Current Army definition of Systems Engineer 


Review of current SE policies Army SE overview NDIA SE conference 


a] How are Systems Engineers currently utilized Review of New Acq Initiatives and implementation for SE, OSD OSD policy on Army SE policy for PEOs 
; : : ' Review of current Army policy for hiring Sys Engrs Definition of policies 
a eee 
Systems Engineers i. _ - . 
Retention theories Review of current literature 


Leadership?/Budget/Stovepipe programs that lack required interoperability |Look at programs that have recently failed, been restructured, or cancelled 
3 
5 


to find out what were the factors driving the decisions -- could they have 


been attributed to the lack of systems engineering? 


What is the cultural resistance to Ses? Rand Study 
Why are recently fielded and currently fielding Army systems not Review of ASAALT briefings PEO-I and FFID direction from VCSA. 


4 | What has changed that makes the need for Systems Engineering greater Review of top 5 SE issues in Defense Industry - NDIA task force en ge eae from handout, RAND STUDY, RDECOM information, 


Why are Army acquisition programs failing 





What is diferent about the way NASA or Navy solves a problem andthe {Comparison of courses offered; where is there a gap. Need Raytheon SE 
z A the Army d 5 lan, NASA SE plan, N: SE plan, Rand ‘t, REDECOM SE 

What is being done by ather parts of the Government, DoD), Industry and Ta oa a structure and mission "on the fly" - what impact aon C3T pace pee ieee INCOSE. 
[Academia in the field of SE and the context of their efforts 7 ae eA 7 Ee apeiaes ; : > 


does this have on SE or on routine vs adaptive expertise? NASA, Academia, Also reference Achitecture training for Capability 


; [is SE focus toonamow? | Managers in TRADOC. 

Fo ee nag | ca ac arm 
to agument systems engineering contractor spaces. 
lhow are these wicked problems solved go away, taming wicked problems. 


Is it enough to have "engineering analysis" levy engineering requirements |TRL migration to SRL, Points awarded for solving problems traditionally |NASA analysis - SRL research, Systems view 
on PMs to adjust product when that will have programmatic cost and. outside the PM's lane. 


schedule impacts 
[Are systems engineers to be empowered over the PMs one serene needed up front - poor articulation of benefits causes pee training for oe Requirements tracability impact on SE, post- 
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How do we apply systems thinking in an approach that leverages SE, Find examples of successful programs that think outside the box. 
operates within the contractual/legal language of Acquisition and 
How do we acquire adaptive experts rather than the routine experts that understands the reality of human behavior impacts on the DoD 


10 tas : 
formal engineering education produces procurement process 


Can we develop courseware that challenges people to think outside the ‘Find classes like NPS, DAU, and UCSD. Focus on critical thinking, non- 

box? itional problem solving. 

Are these examples applicable to ARMY? ind examples of successful programs that think outside the box. 
How are organizations structured to allow for the adaptive attan Project, the Wiz Kids, other examples? We need published 
expertise/paradigm shifing to allow for out of the box solutions i ation that SHOWS how Army te-invents itself every year/every wat/ 
ge in leadership. 


1 











integrator/adaptive expert Curriculum, NPS, INCOSE, NASA, Academia 
he marketplace (both govt and industry) -- what 1s the "best of breed” in 
of these areas that would establish a cohesive coursewarefor Systems 





hat are current government practices and policies Positioning the agency, designing and implementing a diversity program, 


: : ‘ . : s 
‘What is required to retain and motivate the adaptive experts/systems 


us| 
integrators - - - 

tional Grid website Building a world-class workforce 
Assoc of State Workforce Board Chairs Building and retaining the aerospace workforce 


ill Examples from NASA be applicable to Army ASA briefs - show narrow focus does NOT matter in regards to rewards 
What are the tools of the of the SE trade and how can they be maximized to] Cross walk these tools between acquisition, ARCIC, ASA(ALT) and DAU SE training (tools) INCOSE "tools" - SAIC "tools" (release required) 
ensure maximum utili industry. 

Ir 


Cultural and policy impediments Co 





aiming commitment 
Okati 








DOC core competencies 
0 What kind of training curnculum do we build for the systems at NASA SE leadership development program 
in 











15 orate resitance to change Rice bowls..... Library book 
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APPENDIX B. DAU SPRDE-SE CERTIFICATION PROGRAM 


A. DAU SPRDE-SE CERTIFICATION GUIDE LEVEL I, LEVEL IT AND 
LEVEL III (DAU, 2011) 


Feature - Systems Engineering 
Certification Guide Level I 


LevellI | Level II | Level III 


Level I Certification Guide 


Type of 


Assignment Representative Activities 


> Plans, organizes, and conducts engineering activities relating to the design, development, fabrication, 
installation, modification, sustainment, and/or analysis of systerms or systems components for a functional 
: Aepere specialty (i.e., reliability and maintainability, systems safety, materials, avionics, structures, propulsion, 
Functional Specialist chemical/biological, human systems interfaces, weapons, etc,), 
> Demonstrates how systems engineering technical processes and technical management processes guide 
engineering activities for a functional specialty, 


> Plans, organizes, and conducts engineering activities relating to the design, development, and/or analysis 
Software/IT of software and information technology systems or systems components. 
Engineer > Demonstrates how systems engineering technical processes and technical management processes guide 
software development and/or IT integration activities, 


> Plans, organizes, and conducts engineering design and development activities for systems or systems 
Developmental components. 


Engineer > Demonstrates how systems engineering technical processes and technical management processes quide 
design and development activities. 


Science and > Plans, organizes, and conducts science and technology research and engineering activities supporting 


Technology acquisition programs, projects, or activities, 
(Research Eng or > Demonstrates how systems engineering technical processes and technical management processes guide 
Scientist) science and technology research and engineering activities. 


Core Certification Standards (Required for DAWIA certification.) 


Per ieimeeliiiem > ACQ 101 Fundamentals of Systems Acquisition Management 
aires) M@le-liiineme > SYS 101 Fundamentals of Systerns Planning, Research, Development, and Engineering 


Education > Baccalaureate or graduate degree in a technical or scientific field such as engineering, physics, chemistry, 
biology, mathematics, operations research, engineering management, or computer science 

> 1 year of technical experience in an acquisition position from among the following career fields/paths: 

SPRDE-SE, SPRDE-S&T, IT, TRE, PQM, FE, PM, or LCL 


> Similar experience gained from other government positions or industry is acceptable as long as it meets 
the above standards 


Experience 
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Func Soft/IT Dev 
Spc Eng Eng = 


BCF 102 Fundamentals of Earned Value Management ¢ 
BCF 106 Fundamentals of Cost Analysis 
BCF 107 Applied Cost Analysis (R) 


(Res 
ng/Sci) 


aiaeliaiare) 


ee 


CLE 001 Value Engineering 
CLE 004 Introduction to Lean Enterprise Concepts 


: 
et 


CLE 009 System Safety in Systems Engineering 
CLE 011 Modeling and Simulation for Systems Engineering 
CLE 015 Continuous Process Improvement Familiarization 


CLE 036 Engineering Change Proposals for Engineers 


CLL 011 Performance-Based Logistics 


Education 
D> None specified 
Experience 


> One (1) year of technical experience (in addition to core certification experience 
Notes: 

1 The Core Certification Standards section lists the training, education, and experience REQUIRED for certification at this level. 
2 “(R)” following a course title indicates the course is delivered as resident based instruction. 


3 When preparing your IDP, you and your supervisor should consider the training, education, and experience listed in this Core Plus 
Development Guide if not already completed, 


CLM 013 Work-Breakdown Structure 


qq TES A HET 
STA SSHTET 


SISSS SSS 
BIS I WSs A 
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Feature - Systems Engineering 
Certification Guide Level II 


Levell | Level II | Level ii 


Level II Certification Guide 


Type of Dict Representative Activities 
Assignment 
> Organizes, conducts, and/or monitors engineenng activities in o functional specialty relabng to the design, 
development, fabrication, installation, modification, sustainment, and/or analysis of systems or systems 
components. Analyzes, conducts, and/or monitors engineering achvities in a funcbonal specialty relating to 
a tlimealele-lmelalciets|ieimthe design, development, fabrication, installation, modification, sustainment, and/or analysis of systems or 
systems components. 
> Applies systems engineering technical and technical management processes to a functional specialty in 
IPT environments. 


> Organizes, conducts, and/or monitors engineering activities relating to the design, development, and/or 
Software/Il analysis of software and information technology systems or systems components. 
Engineer > Applies systerns engineering technical and technical management processes to software snd IT 
development. 


> Organizes, conducts, and/or monitors engineenng design ond development activites for systems or 
Developmental systems component. 
Engineer » Applies systems engineering technical and technical management processes during systems 
development. 


o i ° . ® 
Science and > Organizes, conducts, and/or monitors science and technology research and engineering activites 


Technology supporbtng acquisition programs, projects, or acbyities, 
(Research Eng or > Applies systems engineering technical and technical management processes to managing or conducting 
Scientist) science and technology research and engineering activities, 


Core Certification Standards (Required for DaWI4 certification.) 


er ea > ACQ 201A Intermediate Systems Acquisibon, Part 4 
Acquisition Training if ACQ 201B Intermediate Systems Acquisition, Part B (R) 
> SYS 202 Intermediate Systems Planning, Research, Development, and Engineering, Part 1 


Psaltis MNe-liiiiiitM@e > SYS 203 [Intermediate Systems Planning, Research, Development, and Engineering, Part 11 (R) 
> CLE 003 Technical Reviews 


: > Baccalaureate or graduate degree in 4 technical or scientific field such as engineering, physics, chemistry, 
Education bi : . fed : : : 
iology, mathematics, operations research, engineering management, or computer science 
> 2 years of technical experience in an acquisition position, Of thet: 
: > - Atleast 1 year in s SPRDE-SE, SPRDE-PSE, or SPRDE-S&T™ position 
Experience >» - Remainder may come from IT, TRE, PQM, PM, or LCL 


> Similor experience gained from other government positions or industry is acceptable os long as it meets 
the above standards 
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Func Soft/IT Dev (Res 
Spc Eng Eng Eng/Sci) 


| CLB O16 Introduction to Earned Value Management__ CTL | UY UT UT 

CLB 017 Performance Measurement Baseline =a 
| CLC 041 Predictive Analysis and Systems Engineering dT LK | UY UT UT 
es 
| CLM 029 Net-Ready Key Performance Parameter (NR-KPP) | 
| CLM O31 Improved Statement of Work 
| CLM 032 Evolutionary Acquisition || 
| CLM 101 Analysis of Alternatives (AoA) (USAF Process) 
| IRM 202 Intermediate Information Systems Acquisition (R) |] CT | 
| LOG 103 Reliability, Availability, and Maintainability (RAM) dT CUT Td 
| LOG 200 Intermediate Acquisition Logistics, PartA dT | CCU Td 

LOG 204 Configuration Management 
| POM 201A Intermediate Production, Quality, and Manufacturing, PartA |] OY OT | 
| STM 202 Intermediate S&T Management(R) 
| TST 203 Intermediate Test and Evaluation (R) 


=e [Vlora lela 


> Graduate degree in a discipline such as engineering, physics, chemistry, biology, mathematics, operations research, engineering 
Management, or computer science 


Training 


« 


SOIGNGD 
NS 


« 


Experience 
> Two (2) years of technical experience (in addition to core certification experience 


Notes: 
1 The Core Certification Standards section lists the training, education, and experience REQUIRED for certification at this level, 
2 “(R)” following a course title indicates the course is delivered as resident based instruction. 





5 When preparing your IDP, you and your supervisor should consider the training, education, and experience listed in the Core Plus 
Development Guide at this and the lower level(s) if not already completed, 

13 Some continuous learning (CL) modules have been created by extracting lessons in their entirety fromm a training course. If this is the 
case for the CL module(s) identified in the above core certification standards, the course from which the CL module was extracted is 
identified in the “Notes” section of the CL course description and the course can be substituted to meet the certification standard. 
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Feature - Systems Engineering 
Certification Guide Level III 


Levell | Leveltr | Level III 


Level IIIT Certification Guide 


Assignment 


Functional Specialist 


Software/IT 
Engineer 


Developmental 
Engineer 
Science and 
Technology 
(Research Eng or 
Scientist) 


Representative Activities 


> Leads and/or manages engineering activities in a functional specialty relating to the design, development, 
fabrication, installation, modification, sustainment, and/or analysis of systems or systems components. 

> Ensures appropriste systems engineering technical and technical manegement processes are properly 
applied to funchonal specialty activities that support IPT environments. 

> Leads and/or manages engineering activities relating to the design, development, and/or analysis of 
software and information technology systems or systerms components. 


> Ensures appropriate systems engineering processes are properly applied to software development and/or 
IT integration activities. 


> Leads and/or manages design and development acbvities for systems or systems components. 
> Ensures appropriate systems engineering processes are properly applied during systems development. 


> Leads and/or manages science and technology research and engineering activities supporting acquisition 
programs, projects, or activities. 

> Ensures appropriate systems engineering processes are properly applied during science and technology 
activibes, 


Core Certification Standards (Required for DAWA certification.) 


Acquisition Training 


Functional Training 


Education 


Experience 


> Acquisition Training identified at Level IT must have been completed 

> S¥S 302 Technical Leadership in Systems Engineering (R) 

> CLL O08 Designing for Supportability in DoD Systerns 

> Baccalaureate or graduate degree in 6 technical or scientific field such as engineering, physics, chemistry, 
biology, mathematics, operations research, engineering management, or computer science 

> 4 years of technical experience in an acquisition position, Of that: 

>» - Atleast 3 year in a SPRDE-SE, SPRDE-PSE, or SPRDE-S&TM position 

>» - Remainder may come from IT, TRE, PQM, PM, or LCL 


> Similar experience gained from other government positions or industry is acceptable as long as it meets 
the above standards 
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Func Soft/IT Dev 


a iieliniave| Spc. Eng Eng f, 


(Res 
ng/Sci) 
¢ 


: 


CLE 008 Six Sigma: Concepts and Processes 

CLE 021 Technology Readiness Assessments 

CLE 301 Reliability and Maintainability 

CLL 022 Title 10 Depot Maintenance Statute Overview 

CLL 023 Title 10 U.S.C, 2464 Core Statute Implementation 

CLL 024 Title 10 Limitations on the Performance of Depot-Level Maintenance (50/50) 
CLL 025 Depot Maintenance Interservice Support Agreements (DMISA) 
CLM 014 IPT Management and Leadership 

CLM 034 Science and Technology—Lesson from PMT 3524 

LOG 201 Intermediate Acquisition Logistics, Part B (R) 

LOG 235 Performance-Based Logistics, Part 4 

LOG 236 Performance-Based Logistics, Part B (R) 

PMT 251 Program Management Tools Course, Part I 

PMT 256 Program Management Tools Course, Part II 

PMT 3524 Program Management Office Course, Part 4 


PQM 203 Preparation of Commercial Item Description for Engineering and Technical 
Personne 


SAM 301 Advanced Software Acquisition Management (R) 
STM 303 Advanced S&T Management (R) 
TST 303 Advanced Test and Evaluation €R) 


R 


: 


Seas 
A ST 


Ii 
As 


=e [Ulet-huleln 


> Graduate degree in a discipline such as engineering, physics, chemistry, biology, mathematics, operations research, engineering 
Management, or computer science 


Experience 
> Four (4) years of technical experience (in addition to core certification experience 
Notes: 
1 The Core Certification Standards section lists the training, education, and experience REQUIRED for certification at this level. 
2 “(R)” following a course title indicates the course is delivered as resident based instruction, 


5 When preparing your IDP, you and your supervisor should consider the training, education, and experience listed in the Core Plus 
Development Guide at this and the lower level(s) if not already completed, 

13 Some continuous learning (CL) modules have been created by extracting lessons in their entirety from a training course. If this is the 
case for the CL module(s) identified in the above core certification standards, the course from which the CL module was extracted is 
identified in the “Notes” section of the CL course description and the course can be substituted to meet the certification standard. 
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APPENDIX C. DAU SPRDE-PSE CERTIFICATION PROGRAM 


A. DAU SPRDE-PSE CERTIFICATION GUIDE LEVEL I, LEVEL II, AND 
LEVEL III (DAU, 2011) 


CERTIFICATION STANDARDS & CORE PLUS DEVELOPMENT GUIDE 
SPRDE —- PROGRAM SYSTEMS ENGINEER LEVEL! 
Type of Assignment Representative Activities 


@ Demonstrates how systems engineering technical and technical management processes apply to acquisition 

programs. 

@ Interacts with program IPTs regarding the proper application of systems engineering processes. 

@ Develops systems models and work breakdown structures; uses top-down design and bottom-up product 

realization 

@ Demonstrates how systems engineering processes apply while working in 2 program office or user support team 

SIT ae)! el ee SUPPONting in-service, out-ofproduction systems 

Systems Engineer @ Interacts with user suppor teams regarding sustainability and reliabilitymaintainability improvements on fielded 
systems. 


Acquisition Program 
Systems Engineer 


Core Certification Standards {required for DAWIA certification 


Acquisition Training ACQ 104 Fundamentals of Systems Acquisition Management 


@ SYS 101 Fundamentals of Systems Planning, Research, Development, and Engineering 

@ Two 100-level courses must come from the following list: 

@ BCF 102 Fundamentals of Earned Value Management 

@ IRM 101 Basic Information Systems Acquisition 

@ LOG 101 Acquisition Logistics Fundamentals 

@ LOG 102 Systems Sustainment Management Fundamentals 

@ PQM 104 Production, Quality, and Manufacturing Fundamentals 

@ TST 102 Fundamentals of Test and Evaluation 

Baccalaureate or graduate degree in a technical or scientific field such as engineering, physics, chemistry, biology, 

mathematics, operations research, engineering management or computer science 

@ 2 years of experience in an SPRDE-SE, SPRDE-PSE. or SPRDE-S&TM acquisition position 

Experience @ Similar expenence gained from other government positions or industry is acceptable as long as it meets the above 
Standards 


Functional Training 


Education 
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Acq Prg Sys Sus Prg Sys 
Eng Eng 
BCF 162 Funcamentais of Eames Value Management 
BCF 106 Funcamentais of Cost Analysis 
BCF 107 Applies Cost Anaiysis (R} 
CLB 009 Piamning, Programming. Busgeting. anc Execution anc Busget Ents 
CLB 016 introduction to Eames Value Management 
CLC 108 Strategic Sourcing Qveniew 
CLC 112 Contractors Accompanying the Force 
CLE 001 Value Engineering 
CLE 004 introduction to Lean Enterprise Concegts 


CLL 002 Detense Logistics Agency Support to he PM 


CLM 621 Introduction to Fecucing Total Ownership Costs (R-TOC) 


[SOR WarnaremecereveRNSEE | 
[ecru roarewevomames SOS 
[aor urseomscmmeee®) SSCS 
[cL Ws Pamang Pogrenng Sages eomneceaeene SSCSC~™Y 
[clbee mammnereVaaWeaee SO OSOSCSCS~SCS 
[clcussesnesumgoeven SSCS 
[clot Coren acmemgreree CSCS 
[cuewivewegeers OSOSOSOSOSOSCSCS 
[cleat manmbancepecnes OOSCSCSCSCS 
[cue beeee ogee ges Samaweern OCS 
[cimewcatemmans———SSSCSCSCSCSOSSCSCSCSCSCSY 
[cmensevesgeee SOSOSOSOSOSOSCSCSCSCSCSCSCS 
cimezt monic Raniang Tes Omewe cue RTO) CSCS 
[Pou Wi pram Gam sclaweargrrerewe OCS 
Tera rnaremecetecoaan SCS 


TST 102 Funcamentai: of Test anc Evaluation 





© The Core Certtication Standards section lists me training equcation anc emerience REQUIRED ‘or certtication at nis leet 
@ “(Ri)” tollowing 2 Course tiie Indicates The course Is Geleres 26 resident based Instruction. 
@ ¥en oregaring your IDP. vou and your suoenicor should consider me training. education. and emerience listed in Tis Core Pus Develooment Guide if not already 


SI 
ele) i 
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CERTIFICATION STANDARDS & CORE PLUS DEVELOPMENT GUIDE 
SPRDE — PROGRAM SYSTEMS ENGINEER LEVEL II 


T 
eS Representative Activities 


Assignment 
Acquisition Program @ Applies systems engineering tecrnical and technical management processes In PTs. 
Systems Engineer @ Develops program/project systems engineering plans, etc. 


Sustainment Program 
Systems Engineer 


Core Certification Standards (required for 0 


Acquisition Training oADDAE buenas ems Aapaaten, Pon (F9 
we Managenet, 


Functional Training 


© TST 203 intermediate Test and Evaluation (R) 


Education Baccalaureate or graduate Cegree In a tecrrical or sclerttc Nels such at engineering. physics. chemistry, Diology. matematics, 
; Operations research, engineering management. or computer science 


© - Atleast 3 years of experience nan SPRDE-SE SPROE-PSE. Of SPROE-SETM aoQuiemion posmion 
@ = Remaincer may come trom IT, TSE POM, PM, or LCL 
© Similar experience gained from COMET QOMETMNENE POBMIONG OF INOUE Ib RODRDMADIE I long a6 R meets Ihe aDOVE STENCETCE 


Experience 
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Acq Prg Sys Sus Prg Sys 
Eng Eng 


[GLEe Sxsgre commmecrmmes SSCS] Sw Sd 
‘SLEAIT TeomcaiPang 
—— 
—— 
Ua A [ 
im wi aren cammanee Mea UDF Pme) ———SSCSCSCSCSCSCSSCS Tw SSCS 
[106 9 Remy Asim woven AA) SSCS] SCS Sr 
CP [ 
[L0G 0 rarmacam Araoummn ogee RETR) ——SSCSCSCSCSSSSSSSCSCSCSCS Tw Si SSS 
a A ( 
[0Gm Peemacetuelogen mae —SSCSSSSOSCCCC Sw Sid SSS 
[eur as Pagan varageret Toe came Pat) —SSCSCSCS~SCSSCSCSCST Sd 
[eMTas Pagan vaageren Toucan Pete ——SSCSCSCSCSSSSSSSSSCSCSCS Tw SC«dS 
[POM 25 earacan Ponisi Gam mevanAamG RRA SSCSCSCSCSCSTSSCSCSCS~CST 
[20M 2918 ramecan Pram Gam wevaaaeng ee) SSCS 
a 


ASAIO CaGFED er GFUEER CEES EARAERNG, PY, EIA. CNN, SEEEENN,CGOREND SERN, GREEN SESNGREEN, CONGR OOD 


Notes: 

© The Core Certification StancRros Section lists the training. equcation anc experience REQUIRED tr certification at tus level 

© “(Ry)” following 2 Course tie Incicates Me COUTEE Is CRIVETES 25 resioent DBSEC INetructiON 

@ When preparing your ID. you an6 your SUPENVISO BOWS CONSIDET Ihe IrBINng. equcBtION anc experience Mstec In Me Core Plus Development Guide at tis anc he 
tower level(s) i not alreaoy completes. 

© Some continuous learning (CL) movies Mave DEEN CrEBIEC Dy Extracting lessons In Trelr entirety trom 2 training course itis Is Me case fOr Me CL moowie(s) lentes 
In he BOVE COME CEmTMMicatiON SIENCRTOS. he Course trom which Ihe CL module was Extracted Is Ientified In he “Notes” section of me Cl course Cescription anc the course 
CBM DE GLOGIRMES 1 meet Me CeNMcation etarcarC 
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CERTIFICATION STANDARDS & CORE PLUS DEVELOPMENT GUIDE 


Type of 
Assignment 


Acquisition Program 
Systems Engineer 
Sustainment Program 


Systems Engineer 


Acquisition Training 


Functional Training 


Education 


Experience 





SPRDE — PROGRAM SYSTEMS ENGINEER LEVEL Ill 


@ Analyzes arc aoolies processes whlle Iegraing mule corals (araiyic or engineering specialties) at a ecten or peters 
Eyeteme vel 
————— Gevelogs ystems engineering piss, cleats and SONses PTE 


finn Stand we ee 
a wal al Ua (Tegured ia 


© & years of experience In an aoquitslion postion OF at: 

© - At least 5 yeare of experience In an SPRDE-SE_ SPRDE-PSE_ or SORDE-SATM acquisiion postin 

@ - Remainder may come trom IT, TEE POM, PML or LCL 

@ Sinfiar experience gained trom ofher goverment poe lions or Industry Is acceptable ac long as K meets he aoe etandarts 
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A ] Pra sys Sus rrq SYS 


Eng Eng2 


Acouemmpneecmamnewomem lw 
oarsmen OS”~=“*‘S*SCS*S*S*S*~*~“~‘“dCSCSSC*‘iCS 
[CLiN on Seem mapmsememge ss) ——SSC~—C~—~—‘—CS*SSsSsC SSS 
fcListaecenee SCS*~“*~S*~“s~*S*SSSST Sd 
a 
[cis 8 Demenng Vanaiy Som aviaera seme CuSIS\nae —~«dtSSCS~—~iCOSCi“ 
fcmaurriementecmers SSS Sid 
[clues twoneu Sm svonpecaanmmnmarsa——S—~sSCCi‘SSC~iCSC 
cumerneemam SSCSC~C—SSSSSSSSOSOSCSTSCt‘“‘“RNCétSC 
[eminemseremegens SC~CSCSSSSSSSSSOSSCSSCSCSCS—S 
Ee 
a PZ PZ 
 eursae reganlesyretonmcarm rate) ——~OC~*~“‘CSCSCSC*~C*~*~*~*~*~é‘dSC‘“‘aySSC*dSOSC‘S 
PZ 
Tamamacrenrceme Cd 


AOA EAA FS SS NN a NO NOD 
retates 





Notes: 

@ The Core Certtication Standards section lists fhe trating. equcation. ac emerience REQUIRED Sr certification 2 fs level 

@ “(FR)” Slowing 2 Course thie Indicates The Courke Is Celered 25 residet Dated Netruction 

@ When preparing your DP. you 350 your eupervacor should Consier Ihe traning. education and experience sted hn he Core Pius Oeveloonent Guide at Inis and ne 
iower levels) fot already completed 

© Some Continous lemming (CL) moovies nave Deen created Oy extracting lessons hn Pel ertirety ‘Tom a training course rile is he Cate Mr Me CL moduie(s) ertiMed 
th he aoove Core CertiMication etancarcs. he courte Tom which Ihe CL module wes extracted iG iectiied ln he “Notes” section of he CL courte Gescrintio and he course 
C27 DE SUDEERUtES 8 meet Ihe Certification stancacc 
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APPENDIX D. ITEMS OF INTEREST THAT EXCEED THE SCOPE 
OF THIS PROJECT 


We have not done any research into the overall cost factor that would be applied 
to any additional training requirements. This would need to be researched in further 


detail prior to implementation of any of the recommendations made in this chapter. 


There has been no analysis done as to the current value of the DAU certification 
offerings as they relate to the Arm’s acquisition needs. There has also been no analysis 
as to the value of the traditional educational formats found in colleges and universities in 
that same context. There would need to be additional analysis done prior to any 


adjustments being made to the above mentioned items. 
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